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Thia  cooMnt  followed  Profeaaor  KoopMn'a 
address  on  ^Fallaclea  in  Operations  Research  at 
the  fourth  annual  aeetlng  of  tne  Operations 
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PROFESSOR  KOQB<AW  OH  FALLACIES  -  A  COMMEfT 
Charle*  Hitch 

I  cannot  itrasa  cnou^  how  fundmatotally  I  aa  In  agraaaent  vlth 
Profeiaor  Koopaan  and  how  auch  I  hope  that  hie  addreae  will  lead  to 
aoaw  reorientation  of  our  thinking  about  OR  and  OR* a  future  developaent. 

Specifically,  I  agree  that  the  aoat  coaaaon  fallaclea  of  OR  occur 
In  connection  with  what  Profeaaor  Koopaan  calla  "choice  of  figurea  of 
aerlt"  and  "choice  of  aeaaurea  of  effort,"  and  which  I  will  refer  to 
Jointly  aa  the  choice  of  criteria,  or  "the  criterion  problea." 

The  extreaely  difficult  problea  of  chooalng  appropriate  criteria 
(l.c.,  teata  of  preferredneaa  which  balance  figurea  of  merit  and  aeaaurea 
of  effort)  la  not  only  the  central  problea  of  GR;  It  la  the  problea  which 
dlatlngulahea  OR  froa  (dare  I  aay?)  other  aclencea.  Operatlona  reaearchera, 
by  definition,  aeek  to  recoaBend  policy,  not  aerely  to  underatand  and 
predict.  In  Profeaaor  Koopaan 'a  worda,  which  I  cannot  laprore  upon,  there 
la  more  to  OR  than  aatheaatlca  and  tha  experlaental  aclencea;  there  la  a 
working  veralon  of  the  concept  of  ralue,  with  all  ita  huaan  and  practical 
OTartonaa . 

Profeaaor  Koopaian  haa  pointed  to  a  nuaber  of  fallaclea  that  arlae  In 
chooalng  criteria.  Inatcad  of  adding  exaaplea,  which  It  la  tempting  to 
do,  let  me  aak:  Why  do  theae  fallaclea  emerge  ao  frequently  in  chooalng 
criteria?  And  let  me  auggeat  aa  a  proximate  anawer;  Becauae  we  have  no 
body  of  theory  to  guide  ua  In  chooalng  good  criteria. 

Why  do  we  have  no  auch  body  of  theory?  .A  aufflclent  explanation  la 
that  we  haven't  yet  had  time  to  develop  one.  OR  la  new.  The  developaient 
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of  good  theory  le  typically  painfully  hard  and  painfully  elow.  What  la 
disturbing  to  m  about  OR,  however,  la  not  the  fact  that  we  haven't  a 
good  body  of  theory  for  chooalng  criteria,  but  that  we  (I  wan  Boat 
practicing  operations  researchers)  don't  sees  Interested  In  developing 
one.  Practicing  operations  researchers,  whether  physical  scientists, 
engineers  or  aatheisatlclans,  with  a  few  exceptions  like  Professor  Koopaan, 
shun  the  whole  problem.  They  don't  even  think  about  It,  let  alone  write 
about  It. 

Why  do  they  shun  It,  when  the  choice  of  criteria  Is  central  to  aost 
OH  probleas,  and  Indeed  the  distinguishing  characteristic  of  ORT  1  suspect 
that  the  underlying  reason  Is  yet  another  fallacy  -  a  variant  of  the  fallacy 
which  philosophers  call  "scientism"  -  a  conviction  In  this  case  that  phil¬ 
osophising  about  criteria,  which  involves  value  Judgments,  Is  "not  science" 
and  therefore  not  quite  respectable.  We  can't  talk  about  It  because  it 
Isn't  science,  even  though  as  long  as  It  remains  "not  science”  OR  can't  be 
science  eltt.er.  Consequently  our  Journals  tend  to  be  filled  with  two 
kinds  of  articles: 

1.  Articles  on  mathematical,  statistical  and  computing  techniques  - 
on  linear  programming,  game  theory,  queuing  theory,  Monte  Carlo, 
etc.  -  eminently  respectable.  In  many  cases  Interesting,  for 

some  problems  very  useful,  but  all  techniques  of  other  disciplines 
and  all  strictly  peripheral  to  OF. 

2.  Case  studies  of  actual  OR,  In  most  of  which  the  figures  of  merit 
and  measures  of  effort  chosen  are  reported  wvth  the  ssuae  casualneas 
with  which  they  were  obviously  selected  In  the  first  place  -  a 
casualneas  which  betrays  the  contempt  of  the  operations  researcher 
for  making  this  choice  at  all. 
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Hov  do  operations  reeearchere  In  fact  eet  about  chooilnf  flgurea  of 
■arit  and  aeasurea  of  effort  -  choice*  to  which  the  results  of  their 
analysis  are  far  acre  sensitive,  In  ay  experience,  than  the  choice  of 
■athsaatlcal  aodel,  to  which  Inordinate  study  and  debate  are  noraally 
devoted!  In  soae  cases,  the  only  and  coaplete  answer  seems  to  be  - 
casually.  The  operations  researcher  takes  the  first  obvious  criterion 
which  pops  Into  his  mind  and  dashes  on  to  the  less  Important  but  more 
confsnlal  aspects  of  hla  Job.  In  othsr  cases,  he  falls  back  on  one  of 

I 

Professor  Koopman's  procedural  fallacies,  either: 

Authorltltls  -  letting  the  customer  choose  the  criterion,  even 
though  this  Involves  letting  the  customer  "ask  the  question"  or 
"define  the  problem"  -  a  thing  no  self-respecting  scientist  will 
let  his  customer  do,  for  good  reasons,  when  he  Is  on  his  own 
familiar  ground  as  scientist.  Or 

Mechanltls  -  putting  nls  machines  to  work  as  a  substitute  for 
hard  thinking.  Because  he  lacks  any  rationale  for  choosing  a 
good  criterion,  the  operation*  researcher  write*  down  all  the 
figures  of  merit  aud  measures  of  effort  which  occur  to  him, 
links  them  In  various  permutations,  and  lets  the  machines  opti¬ 
mise  for  each  In  turn.  Then  either  he  bases  his  recommendations 
on  some  fora  of  majority  vote  (this  might  be  called  the  fallacy 
of  misplaced  democracy  —  all  criteria  are  Inherently  equal),  or 


# 

It  Is  hard  to  find  Instances  In  which  conclusion*  are  not  extremely 
sensitive  to  the  choice  of  criteria.  For  a  recent  familiar  study  In  which 
result*  are  sensitive  see  f.  V.  Hurst,  Jr.,  "Evaluating  the  Adequacy  of 
Airport  Parking  Lots,"  Operations  .^eaearch,  Nov.  1955»  •nd  "Comments"  by 
Stephen  Waldron  and  Jacinto  St«i..hardt,  C^ratlons  Research,  Feb.  19^6. 
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he  coablnct  th«  fallacy  of  authorltltls  with  that  of  sechanitis, 
paaslng  all  reaulta  CQ  to  the  cuatoiter,  who,  even  aore  aubliaely 
Ignorant  of  the  iapllcatlona  of  wt^t  he  la  doing  and  nov  sore 
confuted  than  ceer  by  a  welter  of  aeealngly  e(}ually  plaualble 
opposed  aolutlona,  aakea  the  choice. 

There  le  no  excuse  for  thle  eacaplaa  on  the  part  of  the  operatlona 
reaearcher  froa  the  core  problea  of  hla  dlaclpllne.  I  repeat  that  we 
have  no  theory  to  guide  ua  In  chooalng  good  criteria,  In  the  aenae  of  a 
well-rounded,  ayateaatlc,  widely  accepted  corpua  of  theory.  Jut  we  hare 
a  lot  of  relevant  theory,  much  of  It  good  and  widely  accepted,  which, 
for  exaaple,  peralta  Profeaaor  Koopaan  to  apot  fallacies  In  chooalng 
criteria.  We  don’t  know  how  to  chooee  optlaal  criteria,  but  we  Know 
enough  In  aany  caaea  to  dlatlngulah  the  good  from  the  very  bad. 

Indeed,  there  la  more  theory  In  this  doaaln  than  Profeasor  Koopaan 
uses  In  hla  discussion.  Let  me,  asking  hla  apology  and  understanding, 
develop  my  theses  by  exploiting  his  examples. 

First,  consider  the  antl-subaarlne  warfare  example.  It  Is  clarifying 
and  useful  to  distinguish  the  aggressive  from  the  defensive  use  of  antl- 
aubaarlne  warfare  foraatlons  and  the  very  different  figures  of  serlt 
appropriate  to  each.  But  it  Is  hard  to  think  of  any  pracMcal  ASW 
problem  In  which  one  la  not  Interested  to  some  degree  In  both  the  offen¬ 
sive  and  defensive  utility  of  the  formations.  How  do  we  choose  figures 
of  merit  to  optimise  the  formations?  Certainly  not  by  first  optimising 
for  offense,  then  for  defense,  and  then  somehow  eoablnlng  the  partial 
optima.  This  would  be  analogous  to,  and  ms  bad  as,  taking  the  expected 
value  of  a  function  of  two  variables  as  the  sane  function  of  the  expected 
values  of  each  variable.  We  proceed  either  by  flr'.dlng  a  criterion  which 
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Includei  both  off*n»lv«  and  d#f«a*lv*  utilities,  reducing  thea  to  soae 
cooaon  Masure,  or,  If  we  can't  find  a  way  of  doing  that,  by  laaglnat Irely 
aaaklng  a  near  doalnant  solution  by  Mthods  suggested  by  gaae  theory. 

The  Latter  alternatWe  aay  well  lead  to  the  Invention  by  the  operations 
researcher  of  format  Iona  not  In  the  original  set  to  be  compared. 

I  aa  sure  that  Profeasor  Koopman  wouldn't  disagree.  I  am  merely 
-islng  hla  example  to  show  that  we  can  carry  the  central  analysis  of  criteria 
beyond  the  point  at  which  he  left  It.  The  operations  researcher  aay  be  able 
to  do  better  than  trace  a  boundary  of  "efficient  points"  and  let 
the  customer  choose  his  preferred  point  on  the  boundary. 

Let  us  now  backtrack  and  take  a  harder  look  at  the  suggested  criterion 
of  defensive  utility.  Is  "the  probability  of  preventing  eneay  submarines 
penetrating  the  screen"  a  good  criterion?  Maybe  It  is  -  for  some  hlglily 
constrained  low-level  probleas.  But  to  dm  It  bears  the  familiar  marks  of 
casual  selection.  Probably  one  can  minimise  penetrations  by  keeping  all 
one's  shipping  and  formations  in  harbor.  If  that  Is  ruled  out  as  a  side 
condition,  the  answer  would  appear  to  be  extremely  dense  screens.  Perhaps 
that  Is  the  right  answer,  but  dense  screens  mean  fcv  screens,  and  with 
few  screens  can  enough  shipping  be  moved  across  the  Atlantic?  Maybe  that 
solution  could  be  constrained  In  soae  way  too,  but  I  think  I  Imve  gone 
far  enough  to  Illustrate  two  points: 

1.  The  criteria  In  a  low-level  problem  must  be  consistent  with 

high-level  criteria  -  In  this  cass,  enhancing  the  flow  of 


I  am  not  saying  that  we  must  convert  low-level  IR  probleme  Into  higher 
level  ones  and  work  only  on  tl oae .  On  ths  contrary,  ths  operations  researcher 
must  cut  his  problems  down  to  workable  size.  The  whole  point  of  Diy  article 
on  sub— optimization  (Operations  R'^e^cji,  May  1955) !•  that  he  can  dc  ao  and 
reach  valid  and  uaeful  conclusions  by  choosing  with  csre  criteria  which  are 
conalatent  with  objectives  at  higher  levels.  He  need  not  £olve  global  prob¬ 
lems  In  sll  their  cosiplexlty  In  order  to  recognize  their  general  features, 
a  happy  clrcumatance  which  makss  useful  OR  possible. 
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war  ahlpplng,  the  baalc  raaaon  for  having  ASW  foraatlona. 

2.  B«vare  of  criteria  which  have  to  ba  rapaata^Iy  hadjtd  by 
conatralnta  to  prevent  thea  froa  giving  abaord  reaulta. 

Finally,  according  to  Profeaaor  Koopnan  (and  I  agree)  aoae  of  the 
aoat  glaring  fallaclea  In  chooalng  criteria  are  cauaed  by  the  tendency 
to  uae  aoney  aa  a  meaaure  of  effort.  I  ayaelf  would  gueaa  that  fallaclea 
reault  even  more  frequently  from  failure  to  uae  money  aa  a  aeaaure  of 
effort  where  It  la  a  more  appropriate  measure  than  any  practical  alter¬ 
native.  It  la  my  lapreaalon  that  aoat  eclentlata  and  englneera  In  OR 
are  111  at  ease  with  aoney  raeaaurea,  and  prefer  alaoat  anything  more 
phyalcal  or  tangible  -  like  man-hours,  tons  of  steel,  or  aggregate  time  of 
waiting  -  so  Better  how  Inappropriate  to  the  problem  at  hand. 

But  counting  the  number  of  errors  on  both  aides  of  a  golden  mean  la 
not  the  point.  The  point  la  that  we  have  a  lot  of  pretty  good  theory  to 
guide  us  In  when  to  uae  and  when  not  to  use  money  as  a  measure  of  effort. 
Money  la  not  a  perfect,  but  In  many  circumstances  an  adequate,  coMon 
measure  of  econoalc  resources.  In  some  OR  problems,  we  need  such  a 
aeasure,  and  aoney  cost  la  appropriate  and  an  order  of  magnitude  superior 
to  the  next  beat  alternative.  In  many  other  OR  problems  (including,  I 
think,  all  the  examples  mentioned  by  Professor  Koopman)  mone}  costs  are 
simply  Irrelevant. 

Let  me  develop  this  Idea  a  little  further.  I  think  most  of  us  suspect 
that  the  US  economy  la  more  efficient  than  the  US  military.  This  Is  likely 
to  be  so,  among  other  reasons,  because  aoney  Is  almost  universally  used  In 
the  economy  to  aeasure  both  effort  and  merit,  whereas  In  the  military  no 
corresponding  measure  Is  available  as  a  comsaDn  denominator.  Where  money 
values  are  used  to  measure  effort  and  merit,  a  weak  but  highly  significant 
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kind  of  optimizing  process  results  in  an  economy,  one  with  which  Adam 
Smith  was  familiar,  and  operations  researchers  ought  to  be  but  generally 

4 

aren't.  By  using  money  measures  in  military  OP,  we  can  simulate  this 
optimizing  process  to  a  limited  degree. 

Don't  misinterpret  me.  I  am  not  arguing  for  the  universal  use  of 
money  measures  in  OR.  They  can  be  overused  and  misapplied.  But  there  is  a 
general  case  for  using  money  measures  of  effort  in  certain  circumstances; 
there  is  relevant  theory  which  is  helpful  and  which  needs  to  be  applied, 
developed  and  extended. 

When  one  reads  much  of  what  is  written  on  criteria,  utility,  value 
theory,  etc.,  one  can  understand  and  sympathize  with  OR  editors  and  program 
chairmen  who  are  unsympathetic  to  the  subject.  But  let's  face  it.  Our 
treatjsent  of  criteria  is  the  weakest  link  in  the  OR  chain,  and  we  can't 
simply  shorten  the  chain,  cutting  out  this  link,  for  it  is  central  to  every 
OR  problem.  OR  Itself  can  be  no  more  "scientific,"  if  that  is  our  objective, 
than  this  link.  This  is  where  we  need  to  develop  theory.  And  the  only  road 
to  good  theory,  ve  know  from  the  history  of  every  other  science,  is  by  way 
of  bad  theory.  So  we'd  better  get  on  with  developing  the  theory  we  ne  d, 
even  though,  for  a  time,  it  is  likely  to  be  bad  as  well  as  distasteful. 

And  in  the  meantime  it  would  improve  the  OR  product  enormously  if  we 
would  give  as  much  attention  to  criteria  as  we  do  to  models  or  computing 
techniques.  At  least  we  can  stop  choosing  them  casually. 


See  M.  W.  Hoag,  "Ihe  Relevance  of  Costs  in  Operations  Research," 
p.  ,  this  issue. 


